ci(live-provers): drop continue-on-error on Tier-2/3 live-test steps#118
Merged
Merged
Conversation
The two "Run live test for ${{ matrix.backend }} (allow-fail while
Wave-X wires up)" steps carried `continue-on-error: true`. Comments
explicitly disclosed the soft-fail was transitional, but the gate
silently masks real prover regressions: a successful Tier-2 / Tier-3
matrix run says "live test PASSED" even when the cargo test step
crashed with a real prover error.
This is the same anti-pattern class as echo-types' N5Falsifier xfail
gate (resolved via PR #136): a self-decaying CI assertion that masks
proof-validity once the underlying state matures. Removing both
soft-fails so Tier-2/3 results are honest.
Provision steps and job-level continue-on-error on best-effort
environment setup are preserved — those are intentional resilience
for apt-installing 30+ heterogeneous provers, not proof-validity
masking.
Refs hygiene audit 2026-05-28 (`standards/adoption-readiness-grades/
AUDIT-FINDINGS-2026-05-28.adoc`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The two "Run live test for ${{ matrix.backend }} (allow-fail while Wave-X wires up)" steps in
.github/workflows/live-provers.ymlcarriedcontinue-on-error: true. The comments explicitly disclosed the soft-fail as transitional, but the gate silently masks real prover regressions: a successful Tier-2 / Tier-3 matrix run reports PASS even when thecargo teststep crashed with a real prover error.This is the same anti-pattern class as echo-types' N5Falsifier xfail gate (resolved via echo-types#136): a self-decaying CI assertion that masks proof-validity once the underlying state matures.
Scope
continue-on-error: truefrom the Tier-2 live-test step (line 280).continue-on-error: truefrom the Tier-3 live-test step (line 403).What is NOT changed
Intentional environment resilience preserved:
continue-on-erroron tier1-guix / tier3 / tier4 / tier4-gpu (best-effort prover installations).continue-on-erroron "Provision ${{ matrix.backend }}" steps (apt installation of 30+ heterogeneous provers is genuinely best-effort).continue-on-erroron the Tier-4-GPU "Run live version check (SKIP if binary absent)" step (version check is best-effort by design).Verification
YAML lints clean. No Agda/Coq/Lean files touched — this is a CI-policy change only.
Refs
Estate prover-CI hygiene audit, recorded at:
`standards/adoption-readiness-grades/AUDIT-FINDINGS-2026-05-28.adoc`.
Test plan
🤖 Generated with Claude Code